Audio signal encoding or decoding

ABSTRACT

Encoding an audio signal is provided wherein the audio signal includes a first audio channel and a second audio channel, the encoding comprising subband filtering each of the first audio channel and the second audio channel in a complex modulated filterbank to provide a first plurality of subband signals for the first audio channel and a second plurality of subband signals for the second audio channel, downsampling each of the subband signals to provide a first plurality of downsampled subband signals and a second plurality of downsampled subband signals, further subband filtering at least one of the downsampled subband signals in a further filterbank in order to provide a plurality of sub-subband signals, deriving spatial parameters from the sub-subband signals and from those downsampled subband signals that are not further subband filtered, and deriving a single channel audio signal comprising derived subband signals derived from the first plurality of downsampled subband signals and the second plurality of downsampled subband signals. Further, decoding is provided wherein an encoded audio signal comprising an encoded single channel audio signal and a set of spatial parameters is decoded by decoding the encoded single channel audio channel to obtain a plurality of downsampled subband signals, further subband filtering at least one of the downsampled subband signals in a further filterbank in order to provide a plurality of sub-subband signals, and deriving two audio channels from the spatial parameters, the sub-subband signals and those downsampled subband signals that are not further subband filtered.

The invention relates to encoding an audio signal or decoding an encodedaudio signal.

Erik Schuijers, Weaner Oomen, Bert den Brinker and Jeroen Breebaart,“Advances in Parametric Coding for High-Quality Audio”, Preprint 5852,114th AES Convention, Amsterdam, The Netherlands, 22-25 Mar. 2003disclose a parametric coding scheme using an efficient parametricrepresentation for the stereo image. Two input signals are merged intoone mono audio signal. Perceptually relevant spatial cues are explicitlymodeled as is shown in FIG. 1. The merged signal is encoded using a monoparametric encoder. The stereo parameters Interchannel IntensityDifference (IHD), the Interchannel Time Difference (ITD) and theInterchannel Cross-Correlation (ICC) are quantized encoded andmultiplexed into a bitstream together with the quantized and encodedmono audio signal. At the decoder side, the bitstream is de-multiplexedto an encoded mono signal and the stereo parameters. The encoded monoaudio signal is decoded in order to obtain a decoded mono audio signalm′ (see FIG. 2). From the mono time domain signal, a de-correlatedsignal is calculated using a filter D yielding perceptualde-correlation. Both the mono time domain signal m′ and thede-correlated signal d are transformed to the frequency domain. Then thefrequency domain stereo signal is processed with the IID, ITD and ICCparameters by scaling, phase modifications and mixing, respectively, ina parameter processing unit in order to obtain the decoded stereo pairl′ and r′. The resulting frequency domain representations aretransformed back into the time domain.

An object of the invention is to provide advantageous audio encoding ordecoding using spatial parameters. To this end, the invention providesan encoding method, an audio encoder, an apparatus for transmitting orstoring, a decoding method, an audio decoder, a reproduction apparatusand a computer program product as defined in the independent claims.Advantageous embodiments are defined in the dependent claims.

According to a first aspect of the invention, an audio signal isencoded, the audio signal including a first audio channel and a secondaudio channel, the encoding comprising subband filtering each of thefirst audio channel and the second audio channel in a complex modulatedfilterbank to provide a first plurality of subband signals for the firstaudio channel and a second plurality of subband signals for the secondaudio channel, downsampling each of the subband signals to provide afirst plurality of downsampled subband signals and a second plurality ofdownsampled subband signals, further subband filtering at least one ofthe downsampled subband signals in a further filterbank in order toprovide a plurality of sub-subband signals, deriving spatial parametersfrom the sub-subband signals and from those downsampled subband signalsthat are not further subband filtered, and deriving a single channelaudio signal comprising derived subband signals derived from the firstplurality of downsampled subband signals and the second plurality ofdownsampled subband signals. By providing a further subband filtering ina subband, the frequency resolution of said subband is increased. Suchan increased frequency resolution has the advantage that it becomespossible to achieve higher audio quality (the bandwidth of a singlesub-band signal is typically much higher than that of critical bands inthe human auditory system) in an efficient implementation (because onlya few bands have to be transformed). The parametric spatial coder triesto model the binaural cues, which are perceived on a non-uniformfrequency scale, resembling the Equivalent Rectangular Bands (ERB)scale. The single channel audio signal can be derived directly from thefirst plurality of downsampled subband signals and the second pluralityof downsampled subband signals. However, the single channel audio signalis advantageously derived from sub-subband signals for those downsampledsubbands that are further subband filtered, in which case thesub-subband signals of each subband are added together to form newsubband signals and wherein the single channel audio signal is derivedfrom these new subband signals and the subbands from the first andsecond plurality of subbands that are not further filtered.

According to another main aspect of the invention, audio decoding of anencoded audio signal is provided, the encoded audio signal comprising anencoded single channel audio signal and a set of spatial parameters, theaudio decoding comprising decoding the encoded single channel audiochannel to obtain a plurality of downsampled subband signals, furthersubband filtering at least one of the downsampled subband signals in afurther filterbank in order to provide a plurality of sub-subbandsignals, and deriving two audio channels from the spatial parameters,the sub-subband signals and the downsampled subband signals for thosesubbands that are not further subband filtered. By providing a furthersubband filtering in a subband, the frequency resolution of said subbandis increased and consequently higher quality audio decoding can bereached.

One of the main advantages of these aspects of the invention is thatparametric spatial coding can be easily combined with Spectral BandReplication (“SBR”) techniques. SBR is known per se from Martin Dietz,Lars Liljeryd, Kristofer Kjörling and Oliver Kunz, “Spectral BandReplication, a novel approach in audio coding”, Preprint 5553, 112^(th)AES Convention, Munich, Germany, 10-13 May 2002, and from Per Ekstrand,“Bandwidth extension of audio signals by spectral band replication”,Proc. 1st IEEE Benelux Workshop on Model based Processing and Coding ofAudio (MPCA-2002), pp. 53-58, Leuven, Belgium, Nov. 15, 2002. Furtherreference is made to the MPEG-4 standard ISO/IEC 144963:2001/FDAM1,JTC1/SC29/WG11, Coding of Moving Pictures and Audio, Bandwidth Extensionwhich describes an audio codec using SBR.

SBR is based on the notion that there is typically a large correlationbetween the low and the high frequencies in an audio signal. As such,the SBR process consists of copying the lower part(s) of the spectrum tothe higher part(s) after which the spectral envelope is adjusted for thehigher part(s) of the spectrum using little information encoded in thebit stream. A simplified block diagram of such an SBR enhanced decoderis shown in FIG. 3. The bit-steam is de-multiplexed and decoded intocore data (e.g. MPEG-2/4 Advanced Audio Coding (AAC)) and SBR data Usingthe core data the signal is decoded at half the sampling frequency ofthe full bandwidth signal. The output of the core decoder is analyzed bymeans of a 32 bands complex (Pseudo) Quadrature Mirror Filter (QMF)bank. These 32 bands are then extended to full bandwidth, i.e., 64bands, in which the High Frequency (HF) content is generated by means ofcopying part(s) of the lower bands. The envelope of the bands for whichthe HF content is generated is adjusted according to the SBR dataFinally by means of a 64 bands complex QMF synthesis bank the PCM outputsignal is reconstructed.

The SBR decoder as shown in FIG. 3 is a so-called dual rate decoder.This means that the core decoder runs at half the sampling frequency andtherefore only a 32 bands analysis QMF bank is used. Single ratedecoders, where the core decoder runs at the fill sampling frequency andthe analysis QMF bank consists of 64 bands are also possible. Inpractice, the reconstruction is done by means of a (pseudo) complex QMFbank. Because the complex QMF filter bank is not critically sampled noextra provisions need to be taken in order to account for aliasing. Notethat in the SBR decoder as disclosed by Ekstrand, the analysis QMF bankconsists of only 32 bands, while the synthesis QMF bank consists of 64bands, as the core decoder runs at half the sampling frequency comparedto the entire audio decoder. In the corresponding encoder however, a 64bands analysis QMF bank is used to cover the whole frequency range.

Although the invention is especially advantageous for stereo audiocoding, the invention is also of advantage to coding signals with morethan two audio channels.

These and other aspects of the invention are apparent from and will beelucidated with reference to the embodiments described hereinafter.

In the drawings:

FIG. 1 shows a block diagram of a unit for stereo parameter extractionas used in a Parametric Stereo (“PS”) encoder;

FIG. 2 shows a block diagram of a unit for the reconstruction of astereo signal as used in a PS decoder;

FIG. 3 shows a block diagram of a Spectral Band Replication (“SBR”)decoder, FIG. 4 shows a block diagram of combined PS and SBR enhancedencoder according to an embodiment of the invention;

FIG. 5 shows a block diagram of combined PS and SBR enhanced decoderaccording to an embodiment of the invention;

FIG. 6 shows an M bands downsampled complex QMF analysis (left) andsynthesis bank (right);

FIG. 7 shows a magnitude response in dB of a prototype filter.

FIG. 8 shows a magnitude responses in dB of the first four out of 64non-downsampled complex modulated analysis filters;

FIG. 9 shows a block diagram of a Q bands filter bank with trivialsynthesis;

FIG. 10 shows a combined magnitude response in dB of a firstnon-downsampled modulated QMF filter and 8 bands complex modulatedfilter bank;

FIG. 11 shows a stylized magnitude response of 4 bands evenly stackedfilter bank (top) and oddly stacked filter bank (bottom) according to anembodiment of the invention;

FIG. 12 shows a 77 bands non-uniform hybrid analysis filter bank basedon 64 bands complex analysis QMF according to an embodiment of theinvention;

FIG. 13 shows a 71 bands non-uniform hybrid analysis filter bank basedon 64 bands complex analysis QMF for use in an audio decoder; and

FIG. 14 shows a block diagram of efficient implementation of the complexmodulated analysis filter bank

The drawings only show those elements that are necessary to understandthe invention.

Combining SBR with PS potentially yields an extremely powerful codec.Both SBR and PS are post-processing algorithms in a decoder consistingof a fairly similar structure, i.e., some form of time to frequencyconversion, processing and finally frequency to time conversion. Whencombining both algorithms, it is required that both algorithms can runconcurrently on e.g. a DSP application. Hence, it is advantageous toreuse as much as possible of the calculated intermediate results of onecodec for the other. In the case of combining PS with SBR this leads toreusing the complex (Pseudo) QMF sub-band signals for PS processing. Ina combined encoder (see FIG. 4) the stereo input signal is analyzed bymeans of two 64 bands analysis filter banks. Using the complex sub-banddomain representation, a PS calculation unit estimates the stereoparameters and creates a mono (sub-band) down-mix is created. This monodown-mix is then fed to an SBR parameter estimation unit Finally themono down-mix is converted back to the time domain by means of a 32bands synthesis filter bank such that it can be coded by the coredecoder (core decoder needs only half the bandwidth).

In the combined decoder as shown in FIG. 5, regardless whether or not adual rate or a single rate system is being used, the full bandwidth (64bands) subband domain signals after envelope adjustment are converted toa stereo set of subband domain signals according to the stereoparameters. These two sets of sub-band signals are finally converted tothe time domain by means of the 64 bands synthesis QMF bank. If onewould just combine PS with SBR, the bandwidth of the lower frequencybands of the QMF filter is larger than what is required for a highquality stereo representation. So, in order to be able to give a highquality representation of the stereo image, a further sub-division ofthe lower sub-band signals is performed according to advantageousembodiments of the invention.

For a better understanding of aspects of the invention, the theorybehind complex QMF sub-band filters is first explained.

QMF Sub-Band Filters

The QMF analysis sub-band filter can be described as following. Given areal valued linear phase prototype filter p(v), an M-band complexmodulated analysis filter bank can be defined by the analysis filters$\begin{matrix}{{{h_{k}(v)} = {{p(v)}\exp\left\{ {{\mathbb{i}}\frac{\pi}{M}\left( {k + {1/2}} \right)\left( {v - \theta} \right)} \right\}}},} & (1)\end{matrix}$for k=0,1, . . . ,M−1. The phase parameter θ is not important for theanalysis that follows, but a typical choice is (N+M)/2, where N is theprototype filter order. Given a real valued discrete time signal x(v),the sub-band signals v_(k)(n) are obtained by filtering (convolution)x(v) with h_(k)(v), and then downsampling the result by a factor M (seeleft hand side of FIG. 6).

A synthesis operation consists of first upsampling the QMF sub-bandsignals with a factor M, followed by filtering with complex modulatedfilters of the type (1), adding up the results and finally taking twicethe real part (see right hand side of FIG. 6). Then near-perfectreconstruction of real valued signals can be obtained by suitable designof a real valued linear phase prototype filter p(v). The magnituderesponse of the prototype filter as used in the SBR system of the MPEG-4standard (refered to above) in case of 64 bands is shown in FIG. 7. Themagnitude responses of the 64 complex modulated analysis filters areobtained by shifting the magnitude response of the prototype filter p(v)by π/M(k+½). Part of these responses is shown in FIG. 8. Note that onlythe positive frequencies are filtered, except for k=0 and k=M−1. As aresult the sub-band signals prior to downsampling are close to beinganalytic, facilitating easy amplitude and phase modifications ofreal-valued sinusoids. Phase modifications are also possible for thefirst and last band as long as the sinusoids residing in these bandshave a frequency that is above π/2M or below or π−π/2M respectively. Forfrequencies outside this region the performance of phase modificationdeteriorates rapidly because of interference of the negativefrequencies.

Starting from the QMF analysis filters as described above, inembodiments of the invention, a finer frequency resolution is obtainedby further filtering each downsampled subband signal v_(k)(n) into Q_(k)sub-subbands. In the following the properties of the further subbandfiltering will be derived.

Signal Modification in the Complex QMF Sub-Band Domain

In the following, let Z(ω)=Σ_(n=−∞) ^(∞)z(n)exp(−inω) be the discretetime Fourier transform of a discrete time signal z(n). Assuming thenear-perfect reconstruction property as mentioned above and also adesign where P(ω), the Fourier transform of p(v), essentially vanishesoutside the frequency interval [−π/M,π/M], which is the case for theprototype filter p(v) as illustrated above, the next step here is toconsider a system where the sub-band signals v_(k)(n) are modified priorto synthesis. Now, let each sub band k be modified by filtering with afilter B_(k)(ω). With the extending definitionB _(k)(ω)=B _(−1−k)(−ω)* for k<0,   (2)where the star denotes complex conjugation, it can then be shown(neglecting overall delay, assuming a real valued input and a singlerate system) that the resulting system including filter bank synthesiscorresponds to a filtering with the filter $\begin{matrix}{{B(\omega)} = {\sum\limits_{k = {- M}}^{M - 1}{{B_{k}\left( {M\quad\omega} \right)}\quad{{{P\left( {\omega - {{\pi\left( {k + {1/2}} \right)}/M}} \right)}}^{2}.}}}} & (3)\end{matrix}$

According to the hypotheses regarding the properties of P(ω), insertingB_(k)(ω)=1 for all k in (3) leads to B(ω)=1, and a squared sum identityfollows for the shifted prototype filter responses. By choosingreal-valued constants B_(k)(ω)=b_(k)≧0 the system acts as an equalizer,which interpolates the gain values b_(k) at frequencies π(k+½)/M. Theattractive feature is that the overall system is time-invariant, thatis, free of aliasing, in spite of the use of down- and upsampling. Thiswill of course only be true up to the amount of deviation to the statedprototype filter hypotheses.

In order to derive a mono audio signal, additional sub-filtering of thecomplex sub-band signals should not only preserve these properties, butalso extend these properties to manipulation of the filtered sub-bandsignals. Sub-filtering preserving these properties can be performedusing a modification of so-called Mth band filters as known per se fromP. P. Vaidyanathan, “Multirate systems and filter banks”, Prentice HallSignal Processing Series, 1993, sections 4.6.1-4.6.2).

Modulated Filter Banks with Trivial Synthesis

A discrete time signal v(n) can be split into Q different signals by abank of filters with impulse responses g_(q)(n), q=0,1, . . . ,Q−1. Thisis illustrated in FIG. 9. Let the corresponding analysis outputs bey_(q)(n), and consider the trivial synthesis operation $\begin{matrix}{{y(n)} = {\sum\limits_{q = 0}^{Q - 1}{{y_{q}(n)}.}}} & (4)\end{matrix}$Perfect reconstruction, y(n)=v(n), is then obtained by choosing thefilters such that $\begin{matrix}{{{\sum\limits_{q = 0}^{Q - 1}{g_{q}(n)}} = {\delta(n)}},} & (5)\end{matrix}$where δ(n)=1 if n=0, and δ(n)=0 if n≠0. For causal filters, the righthand side of (5) would have to be replaced with δ(n−d) where d is apositive delay, but this straightforward modification is omitted forclarity of exposition.

The filters g_(q)(n) can be chosen as complex modulations of a prototypefilter g(n) through $\begin{matrix}{{g_{q}(n)} = {{g(n)}\exp\quad{\left\{ {{\mathbb{i}}\frac{2\pi}{Q}\left( {q + {1/2}} \right)n} \right\}.}}} & (6)\end{matrix}$In this preferred embodiment of the invention, the filters are oddlystacked (the factor q+½). An advantage of this preferred embodiment willbe explained later. Perfect reconstruction (5) is obtained if and onlyifg(Qn)=δ(n)/Q.   (7)A variation of this is the real-valued cosine modulation as$\begin{matrix}{{{g_{q}(n)} = {{g(n)}\cos\left\{ {\frac{\pi}{Q}\left( {q + {1/2}} \right)n} \right\}}},} & (8)\end{matrix}$with a real-valued prototype filter g(m) satisfyingg(2Qn)=δ(n)/Q.   (9)(This is easily obtained by consideration of g_(q)(n)+g_(Q−l−q)(n) in(6).)Sub-Filtering the Complex-Exponential Modulated Filter Bank

Starting from the QMF analysis filters as described above, a finerfrequency resolution is obtained by further filtering each downsampledsubband signal v_(k)(n) into Q_(k) sub-subbands by using one of themodulated structures (6) or (8) above. Denote the resulting outputsignals y_(q) ^(k)(n), and let g_(q) ^(k)(n) describe the filter bankapplied within sub band k. If Q_(k)=1, there is no filtering and g₀^(k)(n)=δ(n). A typical application example is the case where M=64,Q₀=8, Q_(k)=4 for k=1,2, and Q_(k)=1 for k>2.

The combined effect of the two filter banks from x(v) to y_(q) ^(k)(n)can be described as filtering with filters F_(q) ^(k)(ω) followed bydownsampling by a factor M, whereF _(q) ^(k)(ω)=H _(k)(ω)G _(q) ^(k)(Mω).   (10)If the prototype filter response P(ω) is essentially zero outside theinterval [−ω/M,π/M], which is the case for the SBR analysis filters (seeFIG. 7), then the filter F_(q) ^(k)(ω) has a single nominal centerfrequency defined in the complex modulated case byω_(k,q)=2π(q+Q _(k) s+½)/(MQ _(k)),   (11)where s is a integer chosen such that Q_(k)(k−½)≦2(q+Q_(k)s)+1≦Q_(k)(k+3/2). For example, as illustrated in FIG. 10, if k=0 and Q₀=8, thevalues of ω_(0,0),ω_(0,1), . . . ,ω_(0,7) are$\frac{\pi}{8M}{\left( {1,3,5,7,9,11,{- 3},{- 1}} \right).}$Signal Modification with Non-Uniform Frequency Resolution

The insertion of sub-subband filter banks as described above does notintroduce further downsampling, so the alias-free performance of signalmodification as shown above in the case of complex QMF only, ispreserved. Consider the general combined operation of M-subbandanalysis, further subband filtering by using Q_(k) sub-subbands withinsubband k, filtering of each sub-subband signal y_(q) ^(k)(n) by afilter A_(k,q)(ω), synthesis within each subband k by summation, andfinally synthesis through the M-band synthesis bank. The overalltransfer function of such a system is given by (3) with, for k≧0,$\begin{matrix}{{{B_{k}(\omega)} = {\sum\limits_{q = 0}^{Q_{k} - 1}{{A_{k,q}(\omega)}{G_{q}^{k}(\omega)}}}},} & (12)\end{matrix}$For ω>π/(2M), this gives $\begin{matrix}{{{B(\omega)} = {\sum\limits_{k = 0}^{M - 1}{\sum\limits_{q = 0}^{Q_{k} - 1}{{A_{k,q}\left( {M\quad\omega} \right)}{G_{q}^{k}\left( {M\quad\omega} \right)}{{P\left( {\omega - {{\pi\left( {k + {1/2}} \right)}/M}} \right)}}^{2}}}}},} & (13)\end{matrix}$so the throughput response of the sub-subband (k,q) is G_(q)^(k)(Mω)|P(ω−π(k+½)/M)|². For |ω|≦π/(2M), some care has to be taken dueto (2). In this frequency range it holds thatB(ω)=B ₀(Mω)|P(ω−π/(2M))|² +B ₀(−Mω)*|P(ω+π/(2M))|²   (14)and assuming a real sub-subband prototype filter coefficients, it holdsthatG _(q) ⁰(−ω)*=G _(Q) ₀ _(−1−q) ⁰(ω),   (15)so if the modifying filters are chosen such thatA _(0,q)(−ω)*=A _(0,Q) ₀ _(−1−q)(ω),   (16)then B₀(−Mω)*=B₀(Mω) and the squared sum identity mentioned inconnection with (3) leads to $\begin{matrix}{{B(\omega)} = {{B_{0}\left( {M\quad\omega} \right)} = {\sum\limits_{q = 0}^{Q_{k} - 1}{{A_{0,q}\left( {M\quad\omega} \right)}{G_{q}^{0}\left( {M\quad\omega} \right)}}}}} & (17)\end{matrix}$for |w|≦π/(2M), corresponding to a throughput response G_(q) ⁰(Mω) forsub-subband (0,q).

Equations (15) until (17) indicate the desire to discriminate betweenpositive and negative frequencies. This is the reason why oddly stacked(complex) filters are being used for sub-filtering the QMF subbandsignals instead of evenly stacked (complex) filters (see FIG. 11). Forevenly stacked filters it is not possible to apply phase modificationsof sinusoids residing in the centre filter, i.e., the filter with acentre frequency of zero, as there is no discrimination between positiveand negative frequencies possible. Assuming a prototype filter with aresponse G(ω) band limited to [−2π/Q,2π/Q], with Q the number of bands,for the evenly stacked case the lower limit to which phase modificationscan approximately be applied is 2π/Q, whereas for the oddly stacked casethe lower limit to which phase modifications approximately can beapplied is π/Q.

As mentioned in the introduction, for PS synthesis important specialcases of the above are equalization and phase modification. Forequalization, A_(k,q)(ω)=a_(k,q)≧0 and the condition (16) reduces toa _(0,q) =a _(0,Q) ₀ _(−1−q).   (18)The phase modification case corresponds to A_(k,q)(ω)=exp(iα_(k,q)) inwhich case the condition (16) is satisfied ifα_(0,Q) ₀ _(−1−q)=−α_(0,q).   (19)Stereo Parameter Estimation

The non-uniform complex filter bank, i.e. the QMF bank followed by thefurther subband filtering, as described above, can be applied toestimate the stereo parameters Inter-channel Intensity Differences(IID), Inter-channel Phase Differences (IPD) and Inter-channel CrossCorrelation (ICC) as shown below. Note that in this practicalembodiment, IPD is used as a practically equivalent substitute for theITD as used in the paper of Schuijers et al. In the combined PS encoder(see FIG. 4) the first three complex QMF channels are sub-filtered sothat in total 77 complex-valued signals are obtained (see FIG. 12).

From this point on the 77 complex-valued time-aligned left and rightsub-subband signals are denoted as l_(q) ^(k)(n) and r_(q) ^(k)(n)respectively, accordingly the indexing of y_(q) ^(k)(n).

To estimate the stereo parameters at a certain sub-band sample positionn′ the left, right and non-normalized cross-channel excitation arecalculated as: $\begin{matrix}\begin{matrix}{{e_{l}(b)} = {\sum\limits_{q = q_{l}}^{q_{h}}\quad{\sum\limits_{k = k_{l}}^{k_{h}}\quad{\sum\limits_{n = 0}^{L - 1}\quad{{h^{2}(n)}\left( {{{l_{q}^{k}\left( {{n'} - \frac{L}{2} + 1 + n} \right)}}^{2} + ɛ} \right)}}}}} \\{{e_{r}(b)} = {\sum\limits_{q = q_{l}}^{q_{h}}\quad{\sum\limits_{k = k_{l}}^{k_{h}}\quad{\sum\limits_{n = 0}^{L - 1}\quad{{h^{2}(n)}\left( {{{r_{q}^{k}\left( {{n'} - \frac{L}{2} + 1 + n} \right)}}^{2} + ɛ} \right)}}}}} \\{{e_{R}(b)} = {\sum\limits_{q = q_{l}}^{q_{h}}\quad{\sum\limits_{k = k_{l}}^{k_{h}}\quad{\sum\limits_{n = 0}^{L - 1}\quad{{h^{2}(n)}\left( {{{l_{q}^{k}\left( {{n'} - \frac{L}{2} + 1 + n} \right)}{r_{q}^{k^{*}}\left( {{n'} - \frac{L}{2} + 1 + n} \right)}} + ɛ} \right)}}}}}\end{matrix} & (20)\end{matrix}$

for every stereo bin b, h(n) is the sub-band domain window with lengthL, ε a very small value preventing division by zero (e.g. ε=1e−10) andl_(q) ^(k)(n) and r_(q) ^(k)(n) the left and right sub-subband domainsignals. In case of 20 stereo bins, the summation over k from k_(l) upto and including k_(h) and q from q_(l) up to and including q_(h) goesas shown in Table. Note that the ‘negative’ frequencies (e.g. k=0 withq=4 . . . 7) are not included in the parameter estimation of (20). TABLE1 Start and stop indices of summation over k and q b k_(l) k_(h) q_(l)q_(h) Pass-band frequency region 0 0 0 0 0    0-π/256 1 0 0 1 1 π/256-2π/256 2 0 0 2 2 2π/256-3π/256 3 0 0 3 3 3π/256-π/64 4 1 1 2 2 π/64-3π/128 5 1 1 3 3 3π/128-2π/64 6 2 2 0 0  2π/64-5π/128 7 2 2 1 15π/128-3π/64 8 3 3 0 0  3π/64-4π/64 9 4 4 0 0  4π/64-5π/64 10 5 5 0 0 5π/64-6π/64 11 6 6 0 0  6π/64-7π/64 12 7 7 0 0  7π/64-8π/64 13 8 8 0 0 8π/64-9π/64 14 9 10 0 0  9π/64-11π/64 15 11 13 0 0 11π/64-14π/64 16 1417 0 0 14π/64-18π/64 17 18 22 0 0 18π/64-23π/64 18 23 34 0 023π/64-35π/64 19 35 63 0 0 35π/64-πThe summations to calculate e_(l)(b), e_(r)(b) and e_(R)(b) are alignedsuch that the mid-point of these signals in the summation coincides withthe parameter position, hence the shift by ${- \frac{L}{2}} + 1.$As is clear from Table 1, only sub-subband signals and subband signalswith a positive centre frequency are used for estimating stereoparameters.

The IID, denoted as I(b), the ICC, denoted as C(b) and the IPD, denotedas P(b) for each stereo bin b are calculated as: $\begin{matrix}\begin{matrix}{{I(b)} = {10\quad{\log_{10}\left( \frac{e_{l}(b)}{e_{r}(b)} \right)}}} \\{{C(b)} = \frac{{e_{R}(b)}}{\sqrt{{e_{l}(b)}{e_{r}(b)}}}} \\{{P(b)} = {{\angle e}_{R}(b)}}\end{matrix} & (21)\end{matrix}$The angle in the equation P(b)=∠e_(R)(b) is calculated using the fourquadrant arctangent function giving values between −π and π. Dependingon target bit rate and application, these parameters, or a subset ofthese parameters are quantized and coded into the PS part of thebit-stream.Stereo Signal Synthesis

In order to keep the computational costs (in terms of RAM usage) in thedecoder as low as possible a similar analysis structure is used. Howeverthe first band is only partially complex (see FIG. 13). This is obtainedby summation of the middle band pairs G₂ ⁰(ω) and G₅ ⁰(ω) and G₃ ⁰(ω)and G₄ ⁰(ω). Furthermore, the second and the third band are two-bandreal-valued filter banks, which is obtained by summation of the outputof G₀ ^(k)(ω) and G₃ ^(k)(ω), and summation of the output of G₁ ^(k)(ω)and G₂ ^(k)(ω) (see also the discussion in the section about modulatedfilter banks). Using this simplification of the decoder filter-bankstructure, still the discriminative feature between positive andnegative frequencies is maintained by subdivision of the first sub-bandfilter. The decoder analysis filter bank is shown in FIG. 13. Noticethat the indexing of the first QMF filtered (sub-)subband signals issorted according to frequency.

The stereo (sub-)subband signals of a single frame are constructed as:l _(k)(n)=Λ₁₁ s _(k)(n)+Λ₂₁ d _(k)(n)   (22)r _(k)(n)=Λ₁₂ s _(k)(n)+Λ₂₂ d _(k)(n)l _(k)(n)=l _(k)(n)e ^(jPn)   (23)r _(k)(n)=r _(k)(n)e ^(−jPn)with s_(k)(n) the mono (sub-)subband signals, and d_(k)(n) the monode-correlated (sub-)subband signals that are derived from the mono(sub-)subband signals s_(k)(n) in order to account for synthesizing theICC parameters, k=0, . . . ,K−1 the sub-band index (K is the totalnumber of sub-bands, i.e., K=71), QMF sub-band sample index n=0, . . .,N−1 with N the number of sub-band samples of a frame, Λ₁₁, Λ₁₂, Λ₂₁,Λ₂₂ the scale factor manipulation matrices and P_(n) the phase rotationmanipulation matrix. The manipulation matrices are defined as functionof time and frequency and can be derived straightforwardly from themanipulation vectors as described in the MPEG-4 standard ISO/IEC14496-3:2001/FPDAM2, JTC1/SC29/WG11, Coding of Moving Pictures andAudio, Extension 2.

s_(k)(n) is defined according to FIG. 12 as resulting in FIG. 13:s ₀(n)=y ₆ ⁰(n)s ₁(n)=y ₇ ⁰(n)s ₂(n)=y ₀ ⁰(n)s ₃(n)=y ₁ ⁰(n)s ₄(n)=y ₂ ⁰(n)+y ₅ ⁰(n)s ₅(n)=y ₃ ⁰(n)+y ₄ ⁰(n)   (24)s ₆(n)=y ₀ ¹(n)+y ₃ ¹(n)s ₇(n)=y ₁ ¹(n)+y ₂ ¹(n)s ₈(n)=y ₀ ²(n)+y ₃ ²(n)s ₉(n)=y ₁ ²(n)+y ₂ ²(n)s _(k)(n)=y ₀ ^(k−7)(n) k=10 . . . 70

Synthesis of the stereo parameters takes place accordingly the indexingof Table 1. TABLE 1 Parameter indexing table k i(k) Pass-band frequencyregion 0  1* −2π/256-−π/256  1  0* −π/256-0    2  0   0-π/256 3  1 π/256-2π/256 4  2 2π/256-3π/256 5  3 3π/256-π/64  6  5 3π/128-2π/64  7 4 2π/128-3π/128 8  6 4π/128-5π/128 9  7 5π/128-6π/128 10  8 3π/64-4π/6411  9 4π/64-5π/64 12 10 5π/64-6π/64 13 11 6π/64-7π/64 14 12 7π/64-8π/6415 13 8π/64-9π/64 16-17 14  9π/64-11π/64 18-20 15 11π/64-14π/64 21-24 1614π/64-18π/64 25-29 17 18π/64-23π/64 30-41 18 23π/64-35π/64 42-70 1935π/64-π   The synthesis equations thus look like:l _(k)(n)=Λ₁₁(i(k),n)s _(k)(n)+Λ₂₁(i(k),n)d _(k)(n)   (25)r _(k)(n)=Λ₁₂(i(k),n)s _(k)(n)+Λ₂₂(i(k),n)d _(k)(n)l _(k)(n)=l _(k)(n)e ^(jP) ^((i(k),n))r _(k)(n)=r _(k)(n)e ^(−jP) ^((l(k),n))   (26)Note that the sign of Pchanges in the equations above if a * isencountered in the table. This is accordingly equation (19), i.e., theinverse phase rotation has to be applied for the negative frequencies.Efficient Implementation of Modulated Filter Banks with TrivialSynthesis

Given a modulated filter bank with a prototype filter of length L, adirect form implementation would require QL operations per input sample,but the fact that the modulation in (6) is antiperiodic with period Qcan be used to split the filtering into a polyphase windowing of Loperations followed by a transform of size Q for each input sample.Please note that a polyphase representation as such is known from P. P.Vaidyanathan, “Multirate systems and filter banks”, Prentice Hall SignalProcessing Series, 1993, section 4.3). The following provides anadvantagous application of such a polyphase representation according toa preferred embodiment of the invention.

The transform is a DFT followed by a phase twiddle, which is of theorder of Q log₂ Q, when Q is a power of two. So a large saving isobtained in typical cases where L is much larger than log₂ Q . In thereal modulated case (8), antiperiodicity of period 2Q combined witheven/odd symmetries around n=0 and n=Q can again be used for polyphasewindowing, and the transform kernel is a DCT of type III. A detaileddescription for the case of complex modulation is given below.

An effective implementation of the sub-subfiltering, using FFT coreprocessing, may be realized using poly-phase decomposition of theprototype filter followed by modulation. Assume a prototype filter g(n)of order N, where N=mQ and m is a positive integer. This condition isnot restrictive, since a prototype filter of arbitrary order can be zeropadded to fulfill the constraint. The Z-transform of the prototypefilter designed for use in a complex modulated system (6) is$\begin{matrix}{{G(z)} = {\sum\limits_{n = {{- N}/2}}^{N/2}\quad{{g(n)}z^{- n}}}} & (27)\end{matrix}$This may be expressed in poly-phase notation as $\begin{matrix}{{{G(z)} = {\sum\limits_{l = 0}^{Q - 1}\quad{{E_{l}\left( z^{Q\quad} \right)}z^{- l}}}}{where}} & (28) \\{{E_{l}(z)} = {\sum\limits_{n = {N/{({2Q})}}}^{N/{({2Q})}}\quad{{g\left( {{Q\quad n} + l} \right)}z^{- n}}}} & (29)\end{matrix}$All filters of the filterbank are frequency-modulated versions of theprototype filter. The Z-transform of the filter g_(q)(n) is given by$\begin{matrix}{{{G_{q}(z)} = {G\left( {z\quad W^{q + \frac{1}{2}}} \right)}}{where}} & (30) \\{W = {\mathbb{e}}^{{- j}\frac{2\pi}{Q}}} & (31)\end{matrix}$The expression for the output from one filter is $\begin{matrix}\begin{matrix}{{Y_{q}(z)} = {{{G_{q}(z)}{V(z)}} = {{{G\left( {z\quad W^{q + \frac{1}{2}}} \right)}V(z)} =}}} \\{= {{\sum\limits_{l = 0}^{Q - 1}\quad{{E_{l}\left( {- z^{Q}} \right)}{V(z)}z^{- l}W^{\frac{1}{2}}W^{{- q}\quad l}}} =}} \\{= {\sum\limits_{l = 0}^{Q - 1}{{E_{l}\left( {- z^{Q}} \right)}{V(z)}z^{- l}{\mathbb{e}}^{j\frac{\pi}{Q}\quad l}{\mathbb{e}}^{j\frac{2\quad\pi}{Q}q\quad l}}}}\end{matrix} & (32)\end{matrix}$By identifying the components of the last sum, it may be seen that thepoly-phase components process delayed versions of the input signal,which subsequently are multiplied by a complex exponential. Finally, allthe output signals Y_(q)(z), q=0 . . . Q−1, are found by applying aninverse FFT (without scaling factor). FIG. 14 shows the layout for theanalysis filter bank. Since the poly-phase filters in (29) arenon-causal, a proper amount of delay has to be added to all thepoly-phase components.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims. In the claims, any reference signsplaced between parentheses shall not be construed as limiting the claim.The word ‘comprising’ does not exclude the presence of other elements orsteps than those listed in a claim. The invention can be implemented bymeans of hardware comprising several distinct elements, and by means ofa suitably programmed computer. In a device claim enumerating severalmeans, several of these means can be embodied by one and the same itemof hardware. The mere fact that certain measures are recited in mutuallydifferent dependent claims does not indicate that a combination of thesemeasures cannot be used to advantage.

1. A method of encoding an audio signal, the audio signal including afirst audio channel and a second audio channel, the method comprisingthe steps of: subband filtering each of the first audio channel and thesecond audio channel in a complex modulated filterbank to provide afirst plurality of subband signals for the first audio channel and asecond plurality of subband signals for the second audio channel,downsampling each of the subband signals to provide a first plurality ofdownsampled subband signals and a second plurality of downsampledsubband signals, further subband filtering at least one of thedownsampled subband signals in a further filterbank in order to providea plurality of sub-subband signals, deriving spatial parameters from thesub-subband signals and from those downsampled subband signals that arenot further subband filtered, and deriving a single channel audio signalcomprising derived subband signals derived from the first plurality ofdownsampled subband signals and the second plurality of downsampledsubband signals.
 2. A method as claimed in claim 1, wherein for eachsubband that is further subband filtered, the sub-subband signals areadded together after scaling and/or phase rotation to form a new subbandsignal, and wherein the single channel audio signal is derived fromthese new subband signals and the downsampled subband signals that arenot further filtered.
 3. A method as claimed in claim 1, wherein thefurther subband filtering is performed on at least the lowest frequencysubband signal of the first plurality of downsampled subband signals andon the lowest frequency subband signal of the second plurality ofdownsampled subband signals.
 4. A method as claimed in claim 3, whereinthe further subband filtering is further performed on at least the nextlowest frequency subband signal of the first plurality of downsampledsubband signals and on the next lowest frequency subband signal of thesecond plurality of downsampled subband signals.
 5. A method as claimedin claim 4, wherein the number of sub-subbands in the lowest frequencysubband signals is higher than the number of sub-subbands in the nextlowest frequency subband signals.
 6. A method as claimed in claim 1,wherein the further subband filterbank is at least partially a complexmodulated filter bank.
 7. A method as claimed in claim 1, wherein thefurther subband filterbank is at least partially a real valued cosinemodulated filter bank.
 8. A method as claimed in claim 1, wherein thefurther subband filter bank is an oddly stacked filter bank.
 9. A methodas claimed in claim 1, wherein the sub-subband signals are not furtherdownsampled.
 10. A method as claimed in claim 1, wherein the singlechannel audio signal is bandwidth limited and further coded and whereinspectral band replication parameters are derived from the firstplurality of downsampled subband signals and/or the second plurality ofdownsampled subband signals.
 11. An audio encoder for encoding an audiosignal, the audio signal including a first audio channel and a secondaudio channel, the encoder comprising: a first complex modulatedfilterbank for subband filtering the first audio channel to provide afirst plurality of subband signals for the first audio channel, a secondcomplex modulated filterbank for subband filtering the second audiochannel to provide a second plurality of subband signals for the secondaudio channel, means for downsampling each of the subband signals toprovide a first plurality of downsampled subband signals and a secondplurality of downsampled subband signals, a further filterbank forfurther subband filtering at least one of the downsampled subbandsignals in order to provide a plurality of sub-subband signals, meansfor deriving spatial parameters from the sub-subband signals and fromthose downsampled subband signals that are not further subband filtered,and means for deriving a single channel audio signal comprising derivedsubband signals derived from the first plurality of downsampled subbandsignals and the second plurality of downsampled subband signals.
 12. Anapparatus for transmitting or storing an encoded audio signal based onan input audio signal, the apparatus comprising: an input unit toreceive an input audio signal, an audio encoder as claimed in claim 11for encoding the input audio signal to obtain an encoded audio signal, achannel coder to further code the encoded audio signal into a formatsuitable for transmitting or storing.
 13. A method of decoding anencoded audio signal, the encoded audio signal comprising an encodedsingle channel audio signal and a set of spatial parameters, the methodof decoding comprising: decoding the encoded single channel audiochannel to obtain a plurality of downsampled subband signals, furthersubband filtering at least one of the downsampled subband signals in afurther filterbank in order to provide a plurality of sub-subbandsignals, and deriving two audio channels from the spatial parameters,the sub-subband signals and those downsampled subband signals that arenot further subband filtered.
 14. A method as claimed in claim 13,wherein the further subband filtering is performed on at least thelowest frequency subband signal of the plurality of downsampled subbandsignals.
 15. A method as claimed in claim 14, wherein the furthersubband filtering is further performed on at least the next lowestfrequency subband signal of the plurality of downsampled subbandsignals.
 16. A method as claimed in claim 15, wherein the number ofsub-subbands in the lowest frequency subband signals is higher than thenumber of sub-subbands in the next lowest frequency subband signals. 17.A method as claimed in claim 13, wherein the further subband filter bankis at least partially a complex modulated filter bank.
 18. A method asclaimed in claim 13, wherein the further subband filterbank is at leastpartially a real valued cosine modulated filter bank.
 19. A method asclaimed in claim 13, wherein the further subband filter bank is an oddlystacked filter bank.
 20. A method as claimed in claim 13, wherein, inthe lowest frequency subband, phase modifications to the sub-subbandsignals having a negative center-frequency in time domain are determinedby taking the negative of the phase modification applied on asub-subband signal having a positive center-frequency which is inabsolute value closest to said negative center-frequency.
 21. A methodas claimed in claim 13, wherein the encoded audio signal comprisesspectral band replication parameters and wherein a high frequencycomponent is derived from the plurality of downsampled subband signalsand the spectral band replication parameters and wherein the two audiochannels are derived from the spatial parameters, the sub-subbandsignals, those downsampled subband signals that are not further subbandfiltered and the high frequency component.
 22. An audio decoder fordecoding an encoded audio signal, the encoded audio signal comprising anencoded single channel audio signal and a set of spatial parameters, theaudio decoder comprising: a decoder for decoding the encoded singlechannel audio channel to obtain a plurality of downsampled subbandsignals, a furher filter bank for further subband filtering at least oneof the downsampled subband signals in a further filterbank in order toprovide a plurality of sub-subband signals, and means for deriving twoaudio channels from the spatial parameters, the sub-subband signals andthose downsampled subband signals that are not further subband filtered.23. An apparatus for reproducing an output audio signal, the apparatuscomprising: an input unit for obtaining an encoded audio signal, anaudio decoder as claimed in claim 22 for decoding the encoded audiosignal to obtain the output audio signal, and a reproduction unit, suchas a speaker or headphone output, for reproducing the output audiosignal.
 24. A computer program product including code for instructing acomputer to perform the steps of the method as claimed in claim
 1. afurther filterbank for further subband filtering at least one of thedownsampled subband signals in order to provide a plurality ofsub-subband signals, means for deriving spatial parameters from thesub-subband signals and from those downsampled subband signals that arenot further subband filtered, and means for deriving a single channelaudio signal comprising derived subband signals derived from the firstplurality of downsampled subband signals and the second plurality ofdownsampled subband signals.
 12. An apparatus for transmitting orstoring an encoded audio signal based on an input audio signal, theapparatus comprising: an input unit to receive an input audio signal, anaudio encoder as claimed in claim 11 for encoding the input audio signalto obtain an encoded audio signal, a channel coder to further code theencoded audio signal into a format suitable for transmitting or storing.13. A method of decoding an encoded audio signal, the encoded audiosignal comprising an encoded single channel audio signal and a set ofspatial parameters, the method of decoding comprising: decoding theencoded single channel audio channel to obtain a plurality ofdownsampled subband signals, further subband filtering at least one ofthe downsampled subband signals in a further filterbank in order toprovide a plurality of sub-subband signals, and deriving two audiochannels from the spatial parameters, the sub-subband signals and thosedownsampled subband signals that are not further subband filtered.
 14. Amethod as claimed in claim 13, wherein the further subband filtering isperformed on at least the lowest frequency subband signal of theplurality of downsampled subband signals.
 15. A method as claimed inclaim 14, wherein the further subband filtering is further performed onat least the next lowest frequency subband signal of the plurality ofdownsampled subband signals.